.TH "Logistic Regression" "" "" "" ""
.SH NAME
.PP
Logistic Regression \- A classification algorithm supported by Frovedis
to predict the binary output with logistic loss.
.SH SYNOPSIS
.PP
\f[C]#include\ <frovedis/ml/glm/logistic_regression_with_sgd.hpp>\f[]
.PP
\f[C]logistic_regression_model<T>\f[]
.PD 0
.P
.PD
logistic_regression_with_sgd::train (\f[C]crs_matrix<T>\f[]& data,
.PD 0
.P
.PD
\  \  \  \ \f[C]dvector<T>\f[]& label,
.PD 0
.P
.PD
\  \  \  \ size_t numIteration = 1000,
.PD 0
.P
.PD
\  \  \  \ T alpha = 0.01,
.PD 0
.P
.PD
\  \  \  \ T miniBatchFraction = 1.0,
.PD 0
.P
.PD
\  \  \  \ T regParam = 0.01,
.PD 0
.P
.PD
\  \  \  \ RegType regtyp = ZERO,
.PD 0
.P
.PD
\  \  \  \ bool isIntercept = false,
.PD 0
.P
.PD
\  \  \  \ T convergenceTol = 0.001,
.PD 0
.P
.PD
\  \  \  \ MatType mType = HYBRID)
.PP
\f[C]logistic_regression_model<T>\f[]
.PD 0
.P
.PD
logistic_regression_with_sgd::train (\f[C]crs_matrix<T>\f[]& data,
.PD 0
.P
.PD
\  \  \  \ \f[C]dvector<T>\f[]& label,
.PD 0
.P
.PD
\  \  \  \ \f[C]logistic_regression_model<T>\f[]& initModel,
.PD 0
.P
.PD
\  \  \  \ size_t numIteration = 1000,
.PD 0
.P
.PD
\  \  \  \ T alpha = 0.01,
.PD 0
.P
.PD
\  \  \  \ T miniBatchFraction = 1.0,
.PD 0
.P
.PD
\  \  \  \ T regParam = 0.01,
.PD 0
.P
.PD
\  \  \  \ RegType regtyp = ZERO,
.PD 0
.P
.PD
\  \  \  \ bool isIntercept = false,
.PD 0
.P
.PD
\  \  \  \ T convergenceTol = 0.001,
.PD 0
.P
.PD
\  \  \  \ MatType mType = HYBRID)
.PP
\f[C]#include\ <frovedis/ml/glm/logistic_regression_with_lbfgs.hpp>\f[]
.PP
\f[C]logistic_regression_model<T>\f[]
.PD 0
.P
.PD
logistic_regression_with_lbfgs::train (\f[C]crs_matrix<T>\f[]& data,
.PD 0
.P
.PD
\  \  \  \ \f[C]dvector<T>\f[]& label,
.PD 0
.P
.PD
\  \  \  \ size_t numIteration = 1000,
.PD 0
.P
.PD
\  \  \  \ T alpha = 0.01,
.PD 0
.P
.PD
\  \  \  \ size_t hist_size = 10,
.PD 0
.P
.PD
\  \  \  \ T regParam = 0.01,
.PD 0
.P
.PD
\  \  \  \ RegType regtyp = ZERO,
.PD 0
.P
.PD
\  \  \  \ bool isIntercept = false,
.PD 0
.P
.PD
\  \  \  \ T convergenceTol = 0.001,
.PD 0
.P
.PD
\  \  \  \ MatType mType = HYBRID)
.PP
\f[C]logistic_regression_model<T>\f[]
.PD 0
.P
.PD
logistic_regression_with_lbfgs::train (\f[C]crs_matrix<T>\f[]& data,
.PD 0
.P
.PD
\  \  \  \ \f[C]dvector<T>\f[]& label,
.PD 0
.P
.PD
\  \  \  \ \f[C]logistic_regression_model<T>\f[]& initModel,
.PD 0
.P
.PD
\  \  \  \ size_t numIteration = 1000,
.PD 0
.P
.PD
\  \  \  \ T alpha = 0.01,
.PD 0
.P
.PD
\  \  \  \ size_t hist_size = 10,
.PD 0
.P
.PD
\  \  \  \ T regParam = 0.01,
.PD 0
.P
.PD
\  \  \  \ RegType regtyp = ZERO,
.PD 0
.P
.PD
\  \  \  \ bool isIntercept = false,
.PD 0
.P
.PD
\  \  \  \ T convergenceTol = 0.001,
.PD 0
.P
.PD
\  \  \  \ MatType mType = HYBRID)
.SH DESCRIPTION
.PP
Classification aims to divide items into categories.
The most common classification type is binary classification, where
there are two categories, usually named positive and negative.
Frovedis supports binary classification algorithm only.
.PP
Logistic regression is widely used to predict a binary response.
It is a linear method with the loss function given by the \f[B]logistic
loss\f[]:
.IP
.nf
\f[C]
L(w;x,y)\ :=\ log(1\ +\ exp(\-ywTx))\ \ \ \ 
\f[]
.fi
.PP
Where the vectors x are the training data examples and y are their
corresponding labels (can be either \-1 for negative response or 1 for
positive response) which we want to predict.
w is the linear model (also called as weight) which uses a single
weighted sum of features to make a prediction.
Logistic Regression supports ZERO, L1 and L2 regularization to address
the overfit problem.
.PP
The gradient of the logistic loss is: \-y( 1 \- 1 / (1 + exp(\-ywTx))).x
.PD 0
.P
.PD
The gradient of the L1 regularizer is: sign(w)
.PD 0
.P
.PD
And The gradient of the L2 regularizer is: w
.PP
For binary classification problems, the algorithm outputs a binary
logistic regression model.
Given a new data point, denoted by x, the model makes predictions by
applying the logistic function:
.IP
.nf
\f[C]
f(z)\ :=\ 1\ /\ 1\ +\ exp(\-z)\ \ 
\f[]
.fi
.PP
Where z = wTx.
By default, if f(wTx) > 0.5, the response is positive (1), else the
response is negative (\-1).
.PP
Frovedis provides implementation of logistic regression with two
different optimizers: (1) stochastic gradient descent with minibatch and
(2) LBFGS optimizer.
.PP
The simplest method to solve optimization problems of the form \f[B]min
f(w)\f[] is gradient descent.
Such first\-order optimization methods well\-suited for large\-scale and
distributed computation.
Whereas, L\-BFGS is an optimization algorithm in the family of
quasi\-Newton methods to solve the optimization problems of the similar
form.
.PP
Like the original BFGS, L\-BFGS (Limited Memory BFGS) uses an estimation
to the inverse Hessian matrix to steer its search through feature space,
but where BFGS stores a dense nxn approximation to the inverse Hessian
(n being the number of features in the problem), L\-BFGS stores only a
few vectors that represent the approximation implicitly.
L\-BFGS often achieves rapider convergence compared with other
first\-order optimization.
.SS Detailed Description
.SS logistic_regression_with_sgd::train()
.PP
\f[B]Parameters\f[]
.PD 0
.P
.PD
\f[I]data\f[]: A \f[C]crs_matrix<T>\f[] containing the sparse feature
matrix
.PD 0
.P
.PD
\f[I]label\f[]: A \f[C]dvector<T>\f[] containing the output labels
.PD 0
.P
.PD
\f[I]numIteration\f[]: A size_t parameter containing the maximum number
of iteration count (Default: 1000)
.PD 0
.P
.PD
\f[I]alpha\f[]: A parameter of T type containing the learning rate
(Default: 0.01)
.PD 0
.P
.PD
\f[I]minibatchFraction\f[]: A parameter of T type containing the
minibatch fraction (Default: 1.0)
.PD 0
.P
.PD
\f[I]regParam\f[]: A parameter of T type containing the regularization
parameter (also called lambda) (Default: 0.01)
.PD 0
.P
.PD
\f[I]regtyp\f[]: A parameter of the type frovedis::RegType, which can be
either ZERO, L1 or L2 (Default: ZERO)
.PD 0
.P
.PD
\f[I]isIntercept\f[]: A boolean parameter to specify whether to include
intercept term (bias term) or not (Default: false)
.PD 0
.P
.PD
\f[I]convergenceTol\f[]: A parameter of T type containing the threshold
value to determine the convergence (Default: 0.001)
.PD 0
.P
.PD
\f[I]mType\f[]: frovedis::MatType parameter specifying the matrix type
to be used for internal calculation (Default: HYBRID for SX
architecture, CRS for other architectures)
.PP
\f[B]Purpose\f[]
.PD 0
.P
.PD
It trains a logistic regression model with stochastic gradient descent
with minibatch optimizer and with provided regularizer (if not ZERO).
It starts with an initial guess of zeros for the model vector and keeps
updating the model to minimize the cost function until convergence is
achieved or maximum iteration count is reached.
After the training, it returns the trained output model.
.PP
\f[B]Return Value\f[]
.PD 0
.P
.PD
After the successful training, it returns a trained model of the type
\f[C]logistic_regression_model<T>\f[].
.SS logistic_regression_with_sgd::train()
.PP
\f[B]Parameters\f[]
.PD 0
.P
.PD
\f[I]data\f[]: A \f[C]crs_matrix<T>\f[] containing the sparse feature
matrix
.PD 0
.P
.PD
\f[I]label\f[]: A \f[C]dvector<T>\f[] containing the output labels
.PD 0
.P
.PD
\f[I]initModel\f[]: A \f[C]logistic_regression_model<T>\f[] containing
the user provided initial model values
.PD 0
.P
.PD
\f[I]numIteration\f[]: A size_t parameter containing the maximum number
of iteration count (Default: 1000)
.PD 0
.P
.PD
\f[I]alpha\f[]: A parameter of T type containing the learning rate
(Default: 0.01)
.PD 0
.P
.PD
\f[I]minibatchFraction\f[]: A parameter of T type containing the
minibatch fraction (Default: 1.0)
.PD 0
.P
.PD
\f[I]regParam\f[]: A parameter of T type containing the regularization
parameter (also called lambda) (Default: 0.01)
.PD 0
.P
.PD
\f[I]regtyp\f[]: A parameter of the type frovedis::RegType, which can be
either ZERO, L1 or L2 (Default: ZERO)
.PD 0
.P
.PD
\f[I]isIntercept\f[]: A boolean parameter to specify whether to include
intercept term (bias term) or not (Default: false)
.PD 0
.P
.PD
\f[I]convergenceTol\f[]: A parameter of T type containing the threshold
value to determine the convergence (Default: 0.001)
.PD 0
.P
.PD
\f[I]mType\f[]: frovedis::MatType parameter specifying the matrix type
to be used for internal calculation (Default: HYBRID for SX
architecture, CRS for other architectures)
.PP
\f[B]Purpose\f[]
.PD 0
.P
.PD
It trains a logistic regression model with stochastic gradient descent
with minibatch optimizer and with provided regularizer (if not ZERO).
Instead of an initial guess of zeors, it starts with user provided
initial model values and keeps updating the model to minimize the cost
function until convergence is achieved or maximum iteration count is
reached.
After the training, it returns the trained output model.
.PP
\f[B]Return Value\f[]
.PD 0
.P
.PD
After the successful training, it returns a trained model of the type
\f[C]logistic_regression_model<T>\f[].
.SS logistic_regression_with_lbfgs::train()
.PP
\f[B]Parameters\f[]
.PD 0
.P
.PD
\f[I]data\f[]: A \f[C]crs_matrix<T>\f[] containing the sparse feature
matrix
.PD 0
.P
.PD
\f[I]label\f[]: A \f[C]dvector<T>\f[] containing the output labels
.PD 0
.P
.PD
\f[I]numIteration\f[]: A size_t parameter containing the maximum number
of iteration count (Default: 1000)
.PD 0
.P
.PD
\f[I]alpha\f[]: A parameter of T type containing the learning rate
(Default: 0.01)
.PD 0
.P
.PD
\f[I]hist_size\f[]: A parameter of size_t type containing the number of
gradient history to be stored (Default: 10)
.PD 0
.P
.PD
\f[I]regParam\f[]: A parameter of T type containing the regularization
parameter (also called lambda) (Default: 0.01)
.PD 0
.P
.PD
\f[I]regtyp\f[]: A parameter of the type frovedis::RegType, which can be
either ZERO, L1 or L2 (Default: ZERO)
.PD 0
.P
.PD
\f[I]isIntercept\f[]: A boolean parameter to specify whether to include
intercept term (bias term) or not (Default: false)
.PD 0
.P
.PD
\f[I]convergenceTol\f[]: A parameter of T type containing the threshold
value to determine the convergence (Default: 0.001)
.PD 0
.P
.PD
\f[I]mType\f[]: frovedis::MatType parameter specifying the matrix type
to be used for internal calculation (Default: HYBRID for SX
architecture, CRS for other architectures)
.PP
\f[B]Purpose\f[]
.PD 0
.P
.PD
It trains a logistic regression model with LBFGS optimizer and with
provided
.PD 0
.P
.PD
regularizer (if not ZERO).
It starts with an initial guess of zeros for the model vector and keeps
updating the model to minimize the cost function until convergence is
achieved or maximum iteration count is reached.
After the training, it returns the trained output model.
.PP
\f[B]Return Value\f[]
.PD 0
.P
.PD
After the successful training, it returns a trained model of the type
\f[C]logistic_regression_model<T>\f[].
.SS logistic_regression_with_lbfgs::train()
.PP
\f[B]Parameters\f[]
.PD 0
.P
.PD
\f[I]data\f[]: A \f[C]crs_matrix<T>\f[] containing the sparse feature
matrix
.PD 0
.P
.PD
\f[I]label\f[]: A \f[C]dvector<T>\f[] containing the output labels
.PD 0
.P
.PD
\f[I]initModel\f[]: A \f[C]logistic_regression_model<T>\f[] containing
the user provided initial model values
.PD 0
.P
.PD
\f[I]numIteration\f[]: A size_t parameter containing the maximum number
of iteration count (Default: 1000)
.PD 0
.P
.PD
\f[I]alpha\f[]: A parameter of T type containing the learning rate
(Default: 0.01)
.PD 0
.P
.PD
\f[I]hist_size\f[]: A parameter of size_t type containing the number of
gradient history to be stored (Default: 10)
.PD 0
.P
.PD
\f[I]regParam\f[]: A parameter of T type containing the regularization
parameter (also called lambda) (Default: 0.01)
.PD 0
.P
.PD
\f[I]regtyp\f[]: A parameter of the type frovedis::RegType, which can be
either ZERO, L1 or L2 (Default: ZERO)
.PD 0
.P
.PD
\f[I]isIntercept\f[]: A boolean parameter to specify whether to include
intercept term (bias term) or not (Default: false)
.PD 0
.P
.PD
\f[I]convergenceTol\f[]: A parameter of T type containing the threshold
value to determine the convergence (Default: 0.001)
.PD 0
.P
.PD
\f[I]mType\f[]: frovedis::MatType parameter specifying the matrix type
to be used for internal calculation (Default: HYBRID for SX
architecture, CRS for other architectures)
.PP
\f[B]Purpose\f[]
.PD 0
.P
.PD
It trains a logistic regression model with LBFGS optimizer and with
provided regularizer (if not ZERO).
Instead of an initial guess of zeors, it starts with user provided
initial model values and keeps updating the model to minimize the cost
function until convergence is achieved or maximum iteration count is
reached.
After the training, it returns the trained output model.
.PP
\f[B]Return Value\f[]
.PD 0
.P
.PD
After the successful training, it returns a trained model of the type
\f[C]logistic_regression_model<T>\f[].
.SH SEE ALSO
.PP
logistic_regression_model, linear_svm
